It is known that there are methods and models to track a three dimensional object in an environment and compute its position and orientation (pose) with respect to a predetermined coordinate system. These kinds of tracker systems are used for example in aircrafts to determine the orientation of head of the pilot. Once the orientation is acquired with respect to the coordinate system of say the display devices, then it is possible to generate graphics on these accordingly. There are different methods to track an object in the scene using magnetic, mechanical or optical means. Currently, the spatial relations of objects may also be determined using magnetic sensors or laser beams but this invention relates specifically to systems using camera-based (day-tv, thermal, IR, Time of Flight etc.) trackers.
In one of the optical camera-based systems the pilot wears a helmet with patterns (fiducial markers) and at least one tracker camera determines the helmet's position and orientation using geometric calculations based on these patterns. Computing spatial relation between an object having a tracking pattern, and a camera is therefore, well known in the state of the art. Throughout the document, whenever a spatial relation is mentioned, it should be understood that the relation between an entity's predetermined reference system with respect to the other's is meant. Determining the position and orientation of an entity using fiducials is called the pose estimation problem and it can be stated as follows: given a set of N feature correspondences between three dimensional (3D) points of an object and two dimensional (2D) projection of that object onto the image plane, find the rotation and translation of the object with respect to the reference system of the camera. The objective is to find rotation and translation between camera and 3D object so that the object's 3D location and orientation is known. This reference system is generally based on the respective pattern of an object under consideration. Since the position and orientation of the tracker camera with respect to the other coordinate systems is known (or can be calculated or measured) in a tracker system, it is also possible to compute the helmet's spatial relation with the tracker camera's sensor and then with other coordinate systems. In this context, “tracked object” means an object having a tracking pattern (fiducial marker) and being tracked by a tracker system. It may be either a helmet as in a helmet-mounted tracker system or any other object.
The patterns used in camera-based tracker systems are either graphical (generally black and white) patterns (passive marker) tracked by visible light cameras or arrays of light sources (e.g. light emitting diodes or LEDs) (active marker). These light sources can be chosen to be in the infrared range of the electromagnetic spectrum with suitable selection of camera sensor and filter set. Other arrangements are also possible but the most convenient among them is the one with the infrared LEDs since these systems can work under inappropriate lighting conditions. The positions (locations) of these LEDs on the tracked object should be determined mindfully to make sure that a small pose error is attained and pose coverage is high. There are some currently used methods to determine and optimize the positions of fiducial markers. In one of such used methods, number of visible fiducials and their relative angle with respect to the optical sensor is used as a constraint to determine optimal fiducial positions. This method is intended to be used in large areas with fiduciary marks and can not be applied to optimize fiducial locations on a moving tracked object being captured by a stationary camera. In addition, when a helmet mounted tracking system is considered, the motion trend of the pilot should also be considered when calculating the fiducial visibility. Furthermore, the pose estimation parameters used by the pose estimation algorithm are not considered in the current methods, which directly affect the output accuracy of the system. There is another work, where motion trend of the pilot and the pose estimation parameters are used by the pose estimation algorithm. However, this method greedily tries to optimize the system, thus it is expected to be stuck at a local optimum solution instead of finding global optimum solution. Furthermore, this method lacks any mechanism to optimize fiducial orientations, which limits its effectivity.
The current methods are not offering an effective way of simulating a tracker system's camera and fiducial positions to optimize the system's pose estimation accuracy altogether. To provide a solution to this problem, a new methodology should be introduced which uses further steps to determine the fiducial positions on a tracked object and positions of cameras in the tracking environment.
The United States patent document US2004239756, an application in the state of the art, discloses a method which uses number of visible fiducials and their relative angle with respect to the capture device as a constraint to determine optimal fiducial positions and compute error bounded pose.
The international patent application numbered PCT/IB12/050801, another application in the state of the art, discloses a method which uses motion trend of the pilot and the pose estimation parameters to determine optimal fiducial positions and compute error bounded pose.
A publication “Felix G Hamza-Lup ET AL: “Marker Mapping Techniques for Augmented Reality Visualization”, COMPUTER AND INFO. SCIENCES, 1 Jan. 2002 (2002-01-01), pages 1-5, XP055087314” discloses two algorithm to distribute markers on complex rigid objects.